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(54) ACTION RECOGNIZING AND PROCESSING DEVICE, AND MOVING OBJECT ACTION 
ANALYZING SYSTEM 

(57)Abstract: 

PROBLEM TO BE SOLVED: To solve such a problem 
that it is hard to catch a moving object moving in a wide 
area with the sufficient resolution and it is hard to 
discriminate the degree of the action of the moving 
body. 

SOLUTION: A parameter input unit 2 inputs the control 
parameter for changing the range of a specified area and 
a concentration value in an image at a predetermined 
rate, and a diffusion computing unit 3 separates the 
moving image of the moving object into an object image 
and the background image, and assigns a predetermined 
concentration value to picture elements of an area in 
which the object image exists at a predetermined time, 
and changes the range of the described area and the 
concentration value with the lapse of time on the basis 
of the control parameter. An action recognizing unit 4 
extracts the feature quantity from the distribution of the 
concentration value in a described area, and recognizes 
the action of the object on the basis of the collation of 
the feature quantity with a specified value. 
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* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

l.This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2 **** s hows the word which can not be translated. 
3. In the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim l]An action recognizing and processing device comprising: 

A parameter inputting part which inputs a control parameter for changing a range and a 

concentration value of a specific field in a picture at a predetermined rate. 

A diffusion calculation part which divides video of an object which moves into an object image and 
a background image, and assigns a predetermined concentration value to a pixel of a field where an 
object image exists in predetermined time and to which a range and the above-mentioned 
concentration value of a described area are changed with the passage of time based on the above- 
mentioned control parameter. 

An identification part of operation which extracts characteristic quantity from distribution of the 
above-mentioned concentration value in a described area, and recognizes operation of the above- 
mentioned object based on collation with the above-mentioned characteristic quantity and a default. 

[Claim 2]In claim 1, when a pixel which is a background image now is an object image before, the 
above-mentioned diffusion calculation part, An action recognizing and processing device assigning a 
concentration value which expresses a negative value to a pixel, assigning a concentration value 
which expresses a positive value to a pixel, and assigning a concentration value expressing a value of 
0 to a changeless portion when a pixel which is an object image now is a background image before. 
[Claim 3] An action recognizing and processing device the above-mentioned diffusion calculation 
part's making a field which assigned the above-mentioned concentration value with the passage of 
time expand in claim 1 or 2, and changing an absolute value of the above-mentioned concentration 
value so that it may decrease. 

[Claim 4] An action recognizing and processing device, wherein the above-mentioned diffusion 
calculation part processes expansion of a field which assigned the above-mentioned concentration 
value by the expansion operation of a picture in claim 1 or 2. 

[Claim 5] An action recognizing and processing device, wherein the above-mentioned diffusion 
calculation part outputs to a display a picture which changed a range and the above-mentioned 
concentration value of a described area, and created them with the passage of time and displays it in 
either of claims 1 thru/or 4. 

[Claim 6]An action recognizing and processing device, wherein a projection concentration value 
which made distribution of the above-mentioned concentration value project on predetermined shaft 
orientations is used for the above-mentioned identification part of operation as the above-mentioned 
characteristic quantity in either of claims 1 thru/or 5. 

[Claim 7] An action recognizing and processing device, wherein it computes a spectrum by carrying 
out the Fourier transform of the above-mentioned identification part of operation from distribution of 
the above-mentioned concentration value and compares with a default by the above-mentioned 
spectrum in either of claims 1 thru/or 6. 

[Claim 8] An action recognizing and processing device, wherein the above-mentioned identification 
part of operation computes a circumscribed rectangle circumscribed to a field over which the above- 
mentioned concentration value is distributed and recognizes operation in either of claims 1 thru/or 7 
based on a ratio of the length of the circumscribed rectangle above-mentioned neighborhood in every 
direction. 
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[Claim 9] An action recognizing and processing device, wherein the above-mentioned identification 
part of operation computes average value of concentration in the predetermined section about 
distribution of the above-mentioned concentration value and recognizes operation in either of claims 
1 thru/or 8 based on size relation of the above-mentioned average value. 

[Claim 10] An action recognizing and processing device judging that a portion which an object 
moves to the above-mentioned section exists when the above-mentioned identification part of 
operation investigates change of concentration in the predetermined section about distribution of the 
above-mentioned concentration value and a bigger change than a predetermined value is detected in 
either of claims 1 thru/or 9. 

[Claim 1 1]A motion analysis system of a movable matter object characterized by comprising the 
following. 

A locus extracting processing part which asks for a moving track of an object which moves and is 
outputted as track data from dynamic image data from a video camera which photoed moving 
operation, such as a person. 

An identification part of operation which discriminates operation of each object which moves from 
the above-mentioned dynamic image data, and is outputted as single operation data. 
A behavior analysis treating part which identifies action as a group and is outputted as group 
operation data from the above-mentioned track data. 

A display processing part which performs a statistical work and three-dimensional-graphics 
processing, and outputs a statistical data and CG picture image data from the above-mentioned single 
operation data and the above-mentioned group operation data. 

[Claim 12]Claim 11 comprising: 

A parameter inputting part which inputs a control parameter for said identification part of operation 
to change a range and a concentration value of a specific field in a picture at a predetermined rate. 
A diffusion calculation part which divides video of an object which moves into an object image and 
a background image, and assigns a predetermined concentration value to a pixel of a field where an 
object image exists in predetermined time and to which a range and the above-mentioned 
concentration value of a described area are changed with the passage of time based on the above- 
mentioned control parameter. 

An identification part of operation which extracts characteristic quantity from distribution of the 
above-mentioned concentration value in a described area, and recognizes operation of the above- 
mentioned object based on collation with the above-mentioned characteristic quantity and a default. 



[Translation done.] 
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DETAILED DESCRIPTION 

[Detailed Description of the Invention] 
[0001] 

[Field of the lnvention]This invention is the art of recognizing operation of a mobile with the 
photoed video, and relates to the suitable operation recognition device for the use which 
supervises or analyzes especially action of a mobile. 
[0002] 

[Description of the Prior Art]The method by an optical flow is known as conventional 
technology which recognizes operation of a person by a picture. This is the technique of 
recognizing operation by regarding change of the image by movement of the object in a 
picture as an optical flow, and analyzing a flow as shown in JP,10-334270,A. 
[0003]As other conventional technologies, as shown in M.l.T.Media Laboratory Perceptual 
Computing Section Technical Report No.402, An objective silhouette is extracted, the 
concentration value according to a temporal change is given to this, the picture called 
Motion History Image (MHI) by integrating a time-axis with it is generated, and there is a 
method of classifying objective operation from the moment characteristic quantity of this 
MHI, etc. 
[0004] 

[Problem(s) to be Solved by the Invention] However, in JP,10-334270,A, since it was the 
technique of having paid one's attention to the concentration distribution of a picture, 
processing when the feature does not exist as a clear pattern of a shade was difficult. Since 
it was difficult to catch a person image in resolution sufficient in a case so that it may be 
targeted at the person who moves especially in a wide area, it was hard to obtain a clear 
shade pattern, and action recognition was also difficult for it. 

[0005]The method of classifying operation was not able to discriminate the grade of the 
operation from the feature of a person's time shape change using the silhouette image. 
There is no consideration about a case so that it may be aimed at the mobile which moves 
in a wide area. 

[0006]The purpose of this invention is to recognize operation of the mobile which moves in 
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a wide area in view of the actual condition of conventional technology. It is also recognizing 

the grade of operation of a mobile. 

[0007] 

[Means for Solving the Problem]ln order to attain the above-mentioned purpose, by a 
parameter inputting part, input a control parameter for changing a range and a 
concentration value of a specific field in a picture at a predetermined rate, and a diffusion 
calculation part, Divide video of an object which moves into an object image and a 
background image, and a predetermined concentration value is assigned to a pixel of a 
field where an object image exists in predetermined time, Changing a range and the above- 
mentioned concentration value of a described area with the passage of time based on the 
above-mentioned control parameter, an identification part of operation extracts 
characteristic quantity from distribution of the above-mentioned concentration value in a 
described area, and recognizes operation of the above-mentioned object based on collation 
with the above-mentioned characteristic quantity and a default. 

[0008]A locus extracting processing part which asks for a moving track of an object which 
moves and is outputted as track data from dynamic image data from a video camera which 
photoed moving operation, such as a person, An identification part of operation which 
discriminates operation of each object which moves from the above-mentioned dynamic 
image data, and is outputted as single operation data, A behavior analysis treating part 
which identifies action as a group and is outputted as group operation data from the above- 
mentioned track data, From the above-mentioned single operation data and the above- 
mentioned group operation data, a statistical work and three-dimensional-graphics 
processing are performed, it has a display processing part which outputs a statistical data 
and CG picture image data, and operation of a movable matter object is analyzed. 
[0009] 

[Embodiment of the lnvention]Hereafter, an embodiment of the invention is described. 
Drawing 1 is the recognition technique of the operation in this invention, and is an 
explanatory view of a cell-like diffusion model. In this invention, since it analyzes with the 
model which the diffusion phenomenon should arise in the surrounding fluid and was 
dispersed in the shape of a cell in that situation by operation of a person, this operation 
recognition technique is called a cell-like diffusion model. Although the kind of operation 
which should be recognized is various, it is aimed at discernment of the moving operation 
of "walking" or "running" here. 

[0010] Drawing 1 (a) is the picture of the person who moves photoed with the video camera 
etc. Although the figure shows only 4x4 pixels, this is a part of neighborhood in which a 
person exists in a picture. Here, the person shall be exactly photoed in the size of stroke 
matter, and a position as shown in a figure shall have a person image at the time of the 
time tO. Since the information on the shade pattern of a person image is not used in this 
invention, the binary picture which a person's silhouette understands may be sufficient as a 
person image. 
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[001 1](b) is what showed the person image in the time t1 , and the person image is moving 
it to the position of the stroke matter right as compared with (a), (c) shows the pressure 
condition of the cell-like diffusion model in t1 . With this model, I think that the portion of 
background images other than the person image of a picture is filled with a certain fluid. 
Although the person is moving to the position of the cell B from the position of the cell A 
now, with this model, it thinks as that in which the person was extinguished in the cell A and 
the person appeared in the cell B. At this time, in the cell A, when the person who existed is 
extinguished, negative pressure tends to arise and a fluid tends to flow by a diffusion 
phenomenon. Similarly, in the cell B, positive pressure tends to arise with the advent of a 
person, and a fluid tends to flow out for diffusion. The pressure of a changeless portion is 0. 

[0012]At this time, as shown in (d), the pressure differential of a fluid will arise in a cell, and 
the disappearance portion of a person like the cell A is made to call a high pressure part for 
depression and the appearance portion of a person like the cell B here. With this figure, the 
hatching portions of ** show depression, as for hatching portions upward slanting to the 
right, the lower right shows a high pressure part, and it means that a pressure differential 
with the circumference is so large that hatching lines are thick. The pressure of the portion 
without hatching is 0. 

[0013](e) shows the diffusion state of the depression in the time t2. in depression, a fluid 
flows by diffusion - the pressure differential with the circumference — small -- becoming (a 
pressure rises) - the field of depression will spread. 

[0014]The pressure of hatching portions is made into the value which applied the 
predetermined value to the pressure value of (d), and stores it in a pixel as a concentration 
value of a picture. It asks for the breadth of a field with the application of the expansion 
operation of image processing. An expansion operation is stroke matter part thicker ****** 
about a person image, stroke matter of the inputted image P is set to Pij, and stroke matter 
of the outputted image Q is set to Qij, and it defines like several 1 as setting a person 
image to 1 and setting a background image to 0 is known well. 
[0015] 

[Equation 1]time of it being 1 Pij or its about eight any they are: Case where they are Qij=1 
and others: Qij=0 — since this expansion operation can be performed by the hardware of 
the image-processing board connected to the personal computer, high-speed processing is 
possible for it. 

[0016](f) shows the diffusion state of a high pressure part similarly. Since a fluid flows out 
by diffusion, a pressure declines and the field of a high pressure part spreads. The 
pressure value of hatching portions is made into the value which subtracted the 
predetermined value from the pressure value of (d), and stores a pressure value in a pixel 
also here. It asks for the spreading field by an expansion operation similarly. 
[0017](g) becomes final pressure distribution [ in / in this / the time t2 ] by the result of 
having added the depression of (e), and the high pressure part of (f). Here, since the 
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absolute value of the pressure value is made into the equal by depression and a high 
pressure part, the pressure value of the field where both exist is 0 by addition. As addition 
of a concentration value which stored the pressure value, addition of these pressure values 
is also performed on an image-processing board. 

[0018]Next, drawing 2 explains the operation recognition technique by a cell-like diffusion 
model. Here, t1 shall have a person image of the time tO in (a), and shall be in (b), (c) and 
t3 shall have t2 in a state of (d), and a person is in a situation which is moving 1 pixel at a 
time to the right. 

[0019]First, as drawing 1 showed, in the time t1, pressure distribution accompanying a 
change of state from (a) to (b) become as shown in (e), are set to (f) by t2, are diffused 
further and set to (g) by t3. By change of (c), pressure distribution consist of (b) like (h) at 
the time t2 similarly, and it becomes (i) in t3. By change of (d), it consists of (c) like (j) at the 
time t3 similarly. 

[0020]ln order to analyze operation of a state from the time tO to t3 here, pressure 
distribution from the above (e) to Q) are added for every time, as the result - (k) - (e) - that 
to which (I) added (f) and (h), and (m) add (g), (i), and Q) as it is. In operation recognition of 
this invention, the feature which a pattern of pressure distribution in each time like (m) has 
from (k) is used. 

r0021] Drawing 3 shows an example of pressure distribution when a person moves. A 
person image is photoed in the state as shown in (a) by part in a picture here, and this 
person is in a situation which is moving in the direction of a right hand of a screen. This 
person image is a silhouette of a person in a picture. 

[0022]Calculating pressure distribution with a technique explained by drawing 2 , the result 
serves as distribution as shown in (b). And a portion in which positive or negative pressure 
exists is extracted, and it asks for the circumscribed rectangle. Since it thinks also when 
two or more persons exist, it distinguishes by carrying out label attachment for every 
circumscribed rectangle in that case. Since a pressure is stored in an image memory as a 
concentration value, a circumscribed rectangle and a result which carried out label 
attachment are obtained using a labeling operation of an image-processing board. 
[0023](c) calculates a pressure projection value of an X axial direction from pressure 
distribution in a circumscribed rectangle. A pressure projection value is what projected a 
pressure value of each cell on specific shaft orientations, and added it, and a value can be 
calculated by specifying a label in a picture and doing a projection operation on an image- 
processing board. 

[0024]When a person moves, it is the usual pattern that a high pressure projection value 
occurs with a convex shape, and a low pressure projection value occurs by a concave in 
the anti-move direction side in the move direction side (in this case, right-hand side) in a 
circumscribed rectangle. It is made to match by a pattern of this pressure projection value, 
and movement of a person is recognized. 

[0025]By drawing 4 , in order to process move recognition, the technique of having applied a 
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Fourier analysis to matching of a pattern is described, (a) is a figure showing distribution of 
a pressure projection value, considers this distribution as a one-dimensional signal, and 
asks for amplitude and a phase spectrum by carrying out discrete Fourier transform of this 
signal. 

[0026]About one-dimensional signal f(i) dispersed now, i= 0, 1 and 2, .., N data of N-1 exist, 

then the Fourier coefficient-C k is expressed by several 2 as known well (j is an imaginary 

unit). 

[0027] 

[Equation 2] 

N-1 

CK=(1/N) I f(i)exp(-j(2»/N)Ki] 
K=0, 1 . 2. • • • • , N — 1 

[0028]A magnitude spectrum and a phase spectrum are called for from this Ck. It is the 
measurement value of (b) which carried out the Fourier transform of the change of the 
pressure projection value of (a), and searched for the spectrum. Although the size of 
pressure distribution changes with positions currently photoed in a screen at this time, the 
size of pressure distribution is normalized so that all the sizes of an X axial direction may 
serve as N pixel here. 

[0029](c) is a spectrum of a person who moves and makes this a reference value. 
Distribution of this spectrum will express characteristic quantity to movement of a person. 
When a spectrum of a measurement value of (b) can judge that it is similar to a reference 
value of (c), a person recognizes as moving. 

[0030]Since a portion of harmonics which a noise at the time of photography tends to 
produce can be cut and compared when judging a signal in a frequency domain like a 
spectrum, there is the strong point of being hard to be influenced by a noise. In the above 
example, a spectrum acquired by matching of a measurement value and a reference value 
with the Fourier transform is used. However, in addition, for example, the residual sum of 
squares of a function of a measurement value and a reference value is calculated about a 
function which shows distribution of a pressure projection value, and both can so judge that 
it is similar that this value is small. 

[0031]Drawing_5_is an explanatory view of operation classification processing. Although it 
can recognize that a person is moving by a method of drawing 4 , it is moving with what 
speed or distinction is impossible. Therefore, it classifies into "it is standing", "he walking", 
or operation of "running" here, using circumscribed rectangle information. 
[0032]lf a person shall move to an X axial direction now, it will become long [ a 
circumscribed rectangle ] to an X axial direction as a case where movement speed is large. 
Since pressure distribution are the shape according to movement magnitude within a time 
[ fixed ], a circumscribed rectangle size will have movement information. And a 
circumscribed rectangle aspect ratio expresses movement speed. 
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[0033]As shown in the figure, Bx and Y shaft-orientations length are set to By for 
circumscribed rectangle X axial direction length, and an aspect ratio is made into Bx/By. In 
this aspect ratio, as shown in (a), when large, the person can judge that it is in a state 
"which is running", and is in a state "stands" in being small as shown in (b), and both middle 
is in a state "which is walking." Even when Y shaft orientations have movement, judgment 
is possible similarly. 

[0034]Next, a flow chart of drawing 6 explains an example of operation recognition 
procedure of this invention. At Step 100, a user inputs a control parameter of diffusion 
calculation using a keyboard or a mouse. It is a parameter which controls a size of a 
concentration value specifically assigned to a pixel, a rate of a concentration value which 
decreases by diffusion, a rate which a field expands by diffusion, etc. Since it is necessary 
to change these according to a photographing condition at the time of recognition, etc., a 
user adjusts here. 

[0035]Step 101 means that only a predetermined frame number repeats processing of Step 
102. Step 102 is a step which creates a background image, between predetermined frame 
numbers, detects a portion which is changeless in a picture, and should just generally make 
the portion a background image. 

[0036]lt means that Step 103 repeats processing of Steps 104-116 about a frame number 
which it is going to measure from now on. At Step 104, a picture for one frame is read into 
an image memory of an image-processing board from a video camera. A person image is 
extracted at Step 105. To this, difference of a background image created by 102 and a 
current image read by 104 is calculated, binarization of this is carried out further, and a 
person's silhouette is obtained as a person image. A background image and a current 
image are stored in an image memory, and these operations can be performed on an 
image-processing board. 

[0037]lt means that Step 106 repeats processing of Steps 107-11 1 about the 
predetermined number of times of diffusion. The number of times of diffusion is the number 
of times of diffusion required for a produced pressure differential to be solved, and is 3, for 
example in an example of drawing 2 here. In this case, supposing the present time is t3, 
tracing back to three frames before, pressure distribution like (m) will be calculated by 
calculating one by one from a picture (a) of to. 

[0038]A high pressure part is extracted at Step 107. In an example of drawing 1 , a high 
pressure part is a portion of a background image at the time to, and is the portion which 
became a person image in t1 . What is necessary is just to carry out a logical operation of a 
picture of (a) and (b) on an image-processing board, in order to ask for this high pressure 
part. At Step 108, depression is extracted similarly. Depression is a portion of a person 
image in to, is the portion which became a background image by t1, and, similarly is 
calculated by a logical operation. 

[0039]A diffusion state of a pressure is searched for at Step 109. As drawing 1 explained, 
signs that a pressure is spread are calculated by the expansion operation of a picture. A 
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size which spreads in diffusion a priori is specified as it sets, for example, 1 pixel expands 
by one diffusion. 

[0040]At Step 1 10, a pressure value which was changed by diffusion is corrected. As 
drawing 1 similarly described, by diffusion, a pressure of depression rises and a pressure of 
a high pressure part decreases. With a value set up a priori also here, it calculates so that a 
predetermined pressure value may be added or subtracted by one diffusion. 
[0041]Pressure distribution are computed at Step 1 1 1 . In an example of drawing 2 , it is 
searching for a picture of (m) and calculates by adding a pressure picture like G) on an 
image-processing board from (e). 

[0042]A pressure projection value is computed at Step 1 12. A pressure projection value is a 
thing as shown in drawing 3 (c), and is calculated by the projection operation of an image- 
processing board from pressure distribution. Discrete Fourier transform is performed at 
Step 113. This is the method which was explained by drawing 4 , and searches for spectrum 
distribution of a pressure projection value. It is a step which compares a spectrum, Step 
114 compares a measurement value with a reference value like drawing 4 , and if similarity 
is high, a person will judge that it is moving. 

[0043]Step 115 is a step which calculates a circumscribed rectangle aspect ratio, and can 
classify operation like drawing 5 like "it running" from the ratio, "he walking", or "standing." 
Step 116 outputs a recognition result. By old processing, since moving operation of a 
person in a picture can be recognized, the result is outputted to a file on a disk, etc. 
[0044] Drawing 7 is a lineblock diagram of an action recognizing and processing device by 
one example. The video camera 7 outputs a video signal to the image-processing board 6. 
Processing is performed in a procedure shown in drawing 6 by the image-processing 
board's 6 having image arithmetic functions, such as an image memory, a logical operation, 
and addition and subtraction, and controlling this by the personal computer 5. 
[0045]The operation recognition treating part 1 is mounted in an inside of the personal 
computer 5 by software or hardware. Here, the parameter inputting part 2 of the operation 
recognition treating part 1 performs Step 100 of a flow of drawing 6 . The diffusion 
calculation part 3 performs processing from Step 101 to Step 111, processes expansion of 
a field which assigned the above-mentioned concentration value with the passage of time 
by the expansion operation of a picture, and it changes an absolute value of the above- 
mentioned concentration value so that it may decrease. And a picture which changed a 
range and the above-mentioned concentration value of a described area, and created them 
with the passage of time is displayed on a display. 

[0046]The identification part 4 of operation performs processing from Step 1 12 to Step 116. 
A spectrum is computed by carrying out the Fourier transform from distribution of the 
above-mentioned concentration value by making into characteristic quantity a projection 
concentration value which made distribution of the above-mentioned concentration value 
project on predetermined shaft orientations, and it compares with a default by the above- 
mentioned spectrum. A circumscribed rectangle circumscribed to a field over which the 
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above-mentioned concentration value is distributed is computed, and operation is 
recognized based on a ratio of the length of the circumscribed rectangle above-mentioned 
neighborhood in every direction. And when change of concentration in the predetermined 
section is investigated about distribution of the above-mentioned concentration value and a 
bigger change than a predetermined value is detected, it is judged that a portion which an 
object moves to the above-mentioned section exists. 

[0047]ln this example, although an image-processing portion is performed by hardware 
called the image-processing board 6, it is also possible to perform all by software in a 
personal computer, without using this. 

[0048]A lineblock diagram at the time of building a system for a person's motion analysis is 
shown in drawing 8 using operation recognition processing by this invention. Dynamic 
image data from the video camera 7 is inputted into the operation recognition processing 1 
and the locus extracting processing 20. 

[0049]ln the locus extracting processing 20, the object extraction part 21 extracts mobiles, 
such as a person in a picture, and an object, the labeling part 22 identifies and pursues 
each mobile, and the 3D coordinate transformation part 23 changes image coordinates into 
three-dimensional space coordinates, and outputs as track data. The contents of this track 
data record a position coordinate on a flat surface where a person existed, for example for 
every time. 

[0050]Since an operation recognition result which the personal computer 1 outputs 
expresses operation of a mobile simple substance, single operation data [ result ] has been 
written here. About a person, an identifier which shows classification of operation of 
"running" or "standing" is recorded here for every time. 

[0051 ]lt is a portion which analyzes a motion as a person's group, and the behavior 
analysis processing 30 consists of the characteristic quantity estimating part 31 which 
calculates characteristic quantity of a motion, and the matching part 32 which compares 
characteristic quantity. The behavior analysis processing 30 outputs group operation data 
which described a meaning of a motion of a group by considering track data as an input. An 
identifier which expresses a meaning of group behavior, such as "evasion" and a "set", as 
an example [ scene / where many persons gather like commercial establishment or a public 
facility about this data ] of security application of person surveillance is recorded for every 
time. 

[0052]The display processing 40 outputs a statistical data in the statistical-analyses part 41 
first by considering single operation data and group operation data as an input. The 
contents of this statistical data are information, including migration length for every 
everybody thing, or the classified number of times of operation, and are displayed on a 
display etc. with CG image. In the following model movement generating section 42, in 
order to move a human body model of three-dimensional CG, it is a portion which defines a 
joint angle of each joint. And in the rendering part 43, CG image is created from a human 
body model, and this is outputted as CG picture image data. 

http://www4.ipdl.inpit.go jp/cgi-bin/tran_web_cgi - ejje?atw_u=http%3A%2F%2Fwww 7/29/2009 



JP,2002-008042,A [DETAILED DESCRIPTION] 



Page 9 of 10 



[0053]Application of those other than security is also available also for a use of game 
analysis in sports, such as soccer. In this case, group operation data will express a 
recognition result of a team play for a recognition result of a motion of a player of each 
[ data / single operation ]. 

[0054]Since it not only can display a person's position, but an identifier of operation 
currently carried out independently can be provided about each person according to this 
example, it is effective in the ability to provide effective information for a person's motion 
analysis. 

[0055]ln the above example, in move recognition processing, although a spectrum was 
calculated from a pressure projection value like drawing 4 and being used as characteristic 
quantity for recognition, other characteristic quantity is available. 
[0056]ln an example of drawing 9 , it recognizes by investigating average value of a 
pressure projection value. Since a mountain of distribution of a pressure projection value is 
certainly generated on right-hand side and a valley of distribution serves as left-hand side 
when a person moves to the right from the left, as shown in the figure (a), this character is 
used. As shown in the figure (b), if distribution of a pressure projection value should be 
acquired from X=Xmin at the section of X=Xmax, center-of-gravity point X=G to this 
distribution shape will be calculated. Average value in right-hand side of PxO and G is set to 
Px1 for average value of a pressure projection value in left-hand side [ G / this ]. If an 
absolute value of PxO and Px1 is Px1>Px0 above a predetermined threshold, a person will 
recognize as moving to the right here. The same may be said of movement of Y shaft 
orientations. 

[0057]Since what is necessary is just to investigate average value of a pressure projection 
value before and behind the center of gravity according to this example, there is the strong 
point in which calculation is easy and processing can be made high-speed. 
[0058]ln the above example, although operation to a motion of a person's whole body was 
recognized, it is also possible to recognize a part where it moved in the body. 
[0059]An example of recognition of a move part is explained to drawing 10 . There shall be 
a pressure projection value which a pressure value by person movement should exist in the 
section of X=Xmin to X=Xmax and the section of Y=Ymin to Y=Ymax, made project on the 
X-axis and Y shaft orientations, and calculated as shown in the figure. The middle point of 
the X-axis and Y shaft orientations is set to Xmid and Ymid here, and a move part of a 
person image is divided and considered in a field like four numbers with a round head 
shown in the figure. A threshold for a judgment should be set to Cx and Cy, and a part 
which an absolute value of a pressure projection value moves to a field beyond these 
values should exist. 

[0060]ln an example of the figure, only a field of a part under the left is moving greatly and it 
is not moving in most other fields. In such a case, it can be recognized as only a person's 
right leg having moved greatly. According to this example, a part greatly moved in the body 
can be known and there is the strong point in which operation can be recognized still more 
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finely, like "having kicked by a right leg" from here. 
[0061] 

[Effect of the lnvention]ln this invention, the silhouette of the movable matter object which 
moves in a wide area is stabilized and obtained, and it is effective in action recognition 
being attained. Since the silhouette-shaped amount of change is changed into 
concentration and memorized, it is effective in the ability to also recognize a grade of 
operation by investigating not only a classification of operation but concentration 
distribution. 
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